Cooperation enhanced by inhomogeneous activity of teaching for evolutionary Prisoner's Dilemma 

games 
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Evolutionary Prisoner's Dilemma games with quenched inhomogeneities in the spatial dynamical rules are 
considered. The players following one of the two pure strategies (cooperation or defection) are distributed on a 
two-dimensional lattice. The rate of strategy adoption from a randomly chosen neighbors are controlled by the 
payoff difference and a two- value pre-factor w characterizing the players whom the strategy learned from. The 
reduced teaching activity of players is distributed randomly with concentrations v at the beginning and fixed 
further on. Numerical and analytical calculations are performed to study the concentration of cooperators as a 
function of w and v for different noise levels and connectivity structures. Significant increase of cooperation is 
found within a wide range of parameters for this dynamics. The results highlight the importance of asymmetry 
characterizing the exchange of master-follower role during the strategy adoptions. 
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Different versions of evolutionary Prisoner's Dilemma 
(PD) games are studied extensively to explore the possibil- 
ities enhancing the cooperative (altruistic) behavior among 
selfish individuals. Originally the Prisoner's Dilemma repre- 
sents those class of two-player symmetric matrix games where 
the equivalent players have two choices [called cooperation 
(C) or defection (D)] with a specific rank of order in the 
four payoffs (dependent on their choices) enforcing both self- 
ish (rational) individuals to choose defection instead of mu- 
tual cooperation yielding significantly higher individual in- 
comes for them LI l2l |2U- This type of social dilemma can- 
not be resolved within the framework of the traditional (two- 
player, one-shot) game theory. During the last decades dif- 
ferent mechanisms, e.g., kin selection |4J], direct |5j] and in- 
direct [6] reciprocity, voluntary participation 01, and spatial 
extensions [8], are found to support the emergence of cooper- 
ative behavior in biological and ecological systems as well as 
within human societies @|. 

In a wide class of the spatial models , the play- 

ers distributed on the sites a; of a lattice can follow one of the 
pure strategies (s x = C or D), their payoffs U x come from 
PD games with their neighbors, and sometimes the players 
are allowed to modify their strategy according to an evolu- 
tionary rule dependent on the local payoff distribution. The 
analogy between these spatial evolutionary games and the 
non-equilibrium kinetic Ising models has motivated a progres- 
sive interest to utilize the tools of non-equilibrium statistical 
physics in the investigation of evolutionary games (for a re- 
view see 0). In these models the connectivity structure of 
the interacting players is described by a graph represented by 
sites (players) and edges between the neighbors. Numerical 
investigations on realistic connectivity structures (e.g., diluted 
11 3l 1 1411 and hierarchical lattice I1 1511 . random graphs Il«.ll7ll . 
small-world structures fl9tl . and real empirical networks 
[20]) have also been performed for some time. The early in- 
vestigations have indicated some increase in the concentration 
of cooperators when the topological inhomogeneities are in- 
creased. Similar phenomena can also be observed if the play- 



ers are allowed to move on a lattice or graph (for a recent 
survey see |21]). The systematic investigations, however, are 
delayed by the large number of parameters characterizing the 
payoffs, the connectivity structures, and the evolutionary rules 
that can involve synchronization and/or noise and be affected 
by the connectivity structure too. 

In the last year Santos et al. Eill observed dramatic 
improvement in the maintenance of cooperation when con- 
sidering an evolutionary PD game on scale-free structures at 
a low noise level. In these systems the competition between 
the cooperative and defective hubs (players with large number 
of neighbors) provides a mechanism 12311 enforcing coopera- 
tion so much so that defectors can die out. The given evo- 
lutionary rule involves intrinsically an inhomogeneous strat- 
egy adoption probability due to the varying number of neigh- 
bors. Remarkable increase of cooperation is also observed for 
those systems where the inhomogeneous imitation activity is 
introduced artificially to characterize the asymmetric and dif- 
ferent influence of players to each other 12411 . In the models 
suggested by Wu et al. I19l 12511 . the influence of player y 
to her neighbor x is quantified by a random parameter w xy 
(0 < w xy ^ w yx < 1) affecting the preferential selection of a 
neighbor whom the strategy can be adopted from. 

Now we study the effect of payoffs and noise on the main- 
tenance of cooperative behavior for some evolutionary PD 
games where the inhomogeneities are involved in the dynam- 
ical rules on regular connectivity structures. Our motivation 
was to explore the consequence of varying activity in the strat- 
egy imitation process. For this purpose, a simplified version 
of the model introduced by Wu et al. 119t 12511 will be con- 
sidered by assuming only two possible values for w xy . First 
our investigation is focused on a system where w xy depends 
only on y and characterizes the teaching (helping) activity of 
player y during the strategy imitations. 

In the first model two types of players {A and B) are dis- 
tributed randomly on a two-dimensional lattice before the start 
of simulation and their distribution (n x = A or B) is fixed 
later on. The concentration of players B and A are denoted 
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by v and (1 — v). Both types of players can follow the C or D 
strategies and their total payoff comes from a PD game with 
the neighbors as formulated in previous papers (see e.g., lEilo . 
For a given two-player game both players receive reward R 
(or punishment P) for mutual cooperation (or defection). If 
the players follow different strategies then the defector re- 
ceives the highest payoff T (temptation to choose defection) 
meanwhile the cooperator gets the lowest (sucker's) payoff S. 
Henceforth our analysis will be restricted to the parametriza- 
tion suggested by Nowak and May |8], i.e., R — 1; P = 0; 
T = b (1< b < 2); and S = 0. 

For the evolutionary PD games the players try to maxi- 
mize their individual payoff by imitating (learning) one of the 
more successful neighboring strategies. Following our pre- 
vious studies lEHal, the evolution of the present system is 
governed by subsequent strategy adoptions between randomly 
chosen neighbors x and y. Namely, player x will adopt the 
neighbor's strategy s y with a probability depending on the 
payoff difference {U x — U y ) as 

WiS ^^^ W ^l + eM(ul~U v )/K} (1) 

where K denotes the amplitude of noise. The pre-factor w xy 
is given as 

f 1, if n v = A 

W *« = \w, if n y = B ' (2) 

where the value of w (0 < w < 1) characterizes the strength 
of reduced teaching activity if the site y is occupied by a 
player of type B. One can think of a system consisting of 
attractive and repulsive players (just like old and young indi- 
viduals in some communities). 

This system is studied by Monte Carlo (MC) simulations 
started from a random initial distribution of C and D strate- 
gies. The stationary state is characterized by the average con- 
centration p of cooperators when the values of K, b, and v are 
varied systematically. First we discuss the MC results on the 
Kagome lattice. The striking increase of p as a result of this 
type of dynamical inhomogeneities is illustrated in Fig. [2 

Evidently, this model is equivalent to a homogeneous sys- 
tem (discussed in j2(J) if w = 1. Furthermore, the average 
concentration of C strategies for v — is equivalent to those 
for v = 1 because the dynamics is homogeneous in both cases 
(the relaxation, however, is slower for v = 1 if w < 1). The 
value of p increases monotonously until reaching the maxi- 
mum value at v w 0.5. For sufficiently small values of w the 
concentration of cooperators reaches its saturation value. Fig- 
ure[Qshows clearly the existence of a parameter range within 
which the defectors become extinct in the stationary states 
(even for b > 1). The rigorous numerical analysis of the tran- 
sition from the two-strategy (C+D) state to the homogeneous 
(absorbing) state is made difficult by the slow relaxation pro- 
cesses characterizing those systems where "directed percola- 
tion" type extinction process |27ll is disturbed by quenched 
disorder in the background 1 28 , 29] . 
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FIG. 1: The concentration of cooperators (p) as a function of v for 
different rates of blocking (from bottom to top, w = 1, 0.2, 0.1, and 
0.05) on the Kagome lattice if b = 1.03 and K = 0.5. Due to the 
large system size (3 • 10 5 < N < 3 ■ 10 6 ) and long sampling times 
the statistical errors of MC data are comparable to the line thickness. 

Now the systematic study is addressed to explore the vari- 
ations in the 6 — K phase diagram caused by the inhomoge- 
neous dynamics. For this purpose we have determined the 
critical values (b cl and 6 c2 ) where the extinction of D and C 
strategies occur when varying b for different noise levels (K). 
Figure |2 compares two phase diagrams obtained for v = 
and v = 0.5 (at w = 0.1). 




FIG. 2: The complete b — K phase diagrams for the Prisoner's 
Dilemma game on the Kagome lattice for v = (left) and 0.5 (right). 
Dashed lines indicate the common limit value of b c \ and b C 2 when 
K — > <x in the homogeneous system. 

Figure |3 illustrates clearly the remarkable difference ap- 
pearing in the high noise limit (K —> oo) while the effect 
of quenched inhomogeneities vanishes if K — > 0. The mean- 
field calculation [12] predicts a step-like change between the 
homogeneous C and D states at 6 = 1. The simulations re- 
produce this prediction for only the homogeneous system in 
the limit K — > oo. According to the simulations the phase 
boundaries {b c i and b C 2) tend towards the same limit b^ (if 
K — > oo), however, the corresponding limit value is larger 
than 1. The increase of b^ (dependent on w and v) cannot be 
described by the mean-field approach. To overcome this short- 
age we have determined the probability of all possible config- 
urations on a triangle of sites allowing the uncorrected distri- 
bution of players A and B (survey of this technique is given 
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in the Appendix of Evaluating 4 3 configuration proba- 
bilities this approach is capable to reproduce qualitatively the 
relevant features as demonstrated in Fig. [5] 
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FIG. 3: Phase boundaries for the evolutionary PD game on the 
Kagome lattice for v = 0.5 and w — 0.1. Solid lines show the 
predictions of the three-site approximation and the symbols indicate 
the MC results. 

In the low noise limit the maintenance of cooperation on the 
Kagome lattice (for b > 1) is due to a mechanism supporting 
the spreading of cooperators throughout overlapping triangles 
of the connectivity structure I2fi Bflll . In the absence of this 
topological feature b C 2 goes to 1 linearly if K — * 0. For ex- 
ample, on the square lattice the cooperation disappear in these 
types of evolutionary PG games if K — > 0. When increasing 
the noise level one can observe a peak in the concentration 
of coo pera tors analogously to the so-called coherence reso- 
nance 

ME!. 

The significant differences in the low noise 
behavior allows us to check the robustness of the observed 
phenomenon. For this purpose the above analysis is repeated 
on the square lattice. 

The MC results on the square lattice (see Fig. |4) illus- 
trates the same trend in the variation of b c \ and b C 2 when the 



quenched inhomogeneity of dynamics is applied (w 
and v = 0.5). 
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FIG. 4: Phase diagram of the evolutionary PD game on the square 
lattice. Closed (open) squares indicate MC results for v — (y = 
0.5). 

The plotted results confirm the above mentioned conclu- 



sions. Namely, the effect of inhomogeneous dynamics van- 
ishes in the zero noise limit and increases the values of b c \ 
and b C 2- It is worth mentioning that the pair approximation 
fails for the homogeneous system in the the low noise limit, 
while its prediction is qualitatively correct for large K values 
j2r3l . In agreement with the expectation, the extended ver- 
sion of the (homogeneous) pair approximation gives account 
of the simultaneous shift of both phase boundaries in a way 
that (b C 2 — b c i) — > if K — > oo. The extended method takes 
explicitly into account the different concentrations of coop- 
erators on the sites of type A and B as well as distinct cor- 
relations on pair of sites AA, BB, and AB. The results of 
MC simulations and extended pair approximation have indi- 
cated clearly that the concentration of cooperators on the sites 
of type B is smaller than those we found on sites of type A 
(shortly ps < Pa) for the coexistence of C and D strategies. 
This latter success of pair approximation has inspired us to in- 
vestigate the changes in the probabilities of one- and two-site 
configurations as well as in the transition rates between these 
configurations. Unfortunately, we were not able to reveal a 
simple and concise explanation on the mechanism supporting 
cooperation. 

Besides the above discussed inhomogeneous dynamics 
there exists many other ways how inhomogeneous (and asym- 
metric) activity can be taken into consideration in the strategy 
adoption process. For example, we have studied what happens 
if the learning activity is reduced for the players of type B. 
In our notation it means that the strategy adoption probabil- 
ity Q is modified by substituting n x for n y in the expression 
(0. Such a model can be used to study the effect of quenched 
spatial disorder on cooperation if the system consist of quick- 
(A) slow-witted (B) players. Surprisingly, MC simulations 
demonstrates that cooperation is not modified relevantly by 
the application of inhomogeneous learning activities on both 
lattice structures for w = 0.1 and v = 0.5. The weak effect 
of this type of inhomogeneous learning activity is justified by 
the analytical techniques too. Namely, the extended pair ap- 
proximation on square lattice (three-site approximation on the 
Kagome lattice) reproduces the result of homogeneous sys- 
tem. 

We have also studied a third type of models for which the 
quenched values of w xy = w yx are chosen randomly to be w 
or 1 with probabilities v and (1 — v). In this case the master- 
follower role is symmetric between the connected players and 
the dynamical inhomogeneity is represented by the randomly 
distributed reduced links. Similarly to the second case the 
simulations indicate no relevant changes in the strategy con- 
centrations. 

To have some further information about the role of the spa- 
tial inhomogeneities and the asymmetry in the imitation pro- 
cess we have studied a fourth case where players A and B 
are located periodically in checkerboard-like manner. In this 
case the role of reduced teaching from the sites of type B is 
equivalent to the reduced learning activity at the sites of type 
A. Furthermore, the interactions are constrained to the A — B 
pairs. The effect of this type of dynamical inhomogeneity on 
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the strategy concentrations depends on the parameter values. 
An increase occurs in p for low values of b. For sufficiently 
large b values the concentration of cooperators (and also b c -i) 
decreases with w. 




FIG. 5: Concentration of cooperators vs. b on the square lattice for 
different systems: homogeneous (closed squares); reduced teaching 
(open squares) and learning (open circles) activity at players B dis- 
tributed randomly, checkerboard-like distribution of players A and 
B (triangles); and randomly distributed w xy — w yx — 0, w (closed 
circles) if v = 0.5, w = 0.1, and K — 1. 

In Figure |5] we illustrate the results of p deduced from MC 
data on the square lattice for the mentioned distributions of 
w xy . The variations can be measured from the results obtained 
on the homogeneous system. It is clearly shown that the most 
relevant enhancement of cooperation is found when the teach- 
ing activity is reduced (or enhanced) from half of the players 
distributed randomly. According to preliminary results this is 
the only model among the investigated systems that is capable 
to maintain cooperation if b > 1 for the high noise limit. 

In summary, we have studied two-strategy evolutionary 
PD games on two-dimensional lattices in which the master- 
follower asymmetry between two neighboring players is taken 
explicitly into account during the strategy adoption mecha- 
nism. Our analysis is focused on several cases of the quenched 
inhomogeneous dynamics. The most important increase of 
cooperativity is found for those systems where the enhanced 
teaching activity is concentrated on a portion of local players 
(masters or leaders) distributed randomly in the whole system. 
It is conjectured that the spontaneous appearance of local lead- 
ers can be recognized for those human and animal societies 
where mimics (learning) plays a crucial role in the mainte- 
nance of cooperation. 

This work was supported by the Hungarian National Re- 
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